Pointer initiated instant bilingual annotation on textual information in an electronic document

ABSTRACT

This invention provides a system and method for providing a user an artificial intelligence based bilingual annotation, displayed in a callout associated with the user&#39;s mouse pointer, on a piece of textual information contained in a segment of text adjacent to, or overlaid by, the user&#39;s mouse pointer while the user is reading an electronic document on the computer screen.

This application claims priority to the U.S. provisional patentapplication Ser. No. 60/414,623, filed on 30 Sep. 2002, the contents ofwhich are incorporated by reference herein.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates generally to machine translation technology. Moreparticularly, the invention relates to a bilingual linguistic annotationcalibration engine (LACE) which comprises a system and method forautomatically returning a user from a local computer or a web server anartificial intelligence based bilingual annotation, displayed in acallout or bubble, on a piece of textual information, such as a phrase,a keyword, or a sentence, contained in a segment of text adjacent to oroverlaid by the user's mouse pointer while the user is viewing anelectronic document on the computer screen.

2. Description of Prior Art

The World Wide Web refers to the complete set of documents residing onall Internet servers that use the HTTP protocol, accessible to users viaa simple point-and-click system. Because the Internet is borderless, anyuser on the earth can access a web site hosted by any web server as longas the devices required for Internet connection are available.

With the broad use of the Internet all over the world, WWW has becomethe primary information resource for many of those who can access theInternet. Web users seek information not only from the web sites intheir own language, but also from the web sites in foreign languages. Toassist the users with different language backgrounds, many site-hostsprovide multilingual versions of their web sites. For example, in orderto attract readers from Western countries, many Chinese, Korean andJapanese web sites include versions in English, German or French.Similarly, to attract Asian readers, some American web sites alsoinclude versions in Chinese, Korean or Japanese.

Although, as a matter of fact, a multilingual web site best serves auser who has bilingual need, from the point of view of the site owners,it is not cost effective. First, it is quite expensive to hireprofessionals to translate the web pages and their updates intodifferent languages. For a large web site with hundreds even thousandsof pages of documents, the project of translation is huge. Second,because the translation takes time, the multilingual versions cannot beupdated in a timely manner. Third, the more versions a web site has, themore inconsistencies there exist among different versions. Sometimescentrality, integrity, or consistency is of essence. Fourth, amultilingual web site not only burdens the host for requiring largerdatabases and higher process capabilities, but also burdens the Internetfor creating heavier traffic.

Therefore, it becomes a need to provide a user a tool or tools to read aweb site which is in a language other than the user's own language.

Ning-Ping Chan et al. have been granted on Aug. 5, 2003 a U.S. Pat. No.6,604,101 for their invention entitled “METHOD AND SYSTEM FORTRANSLINGUAL TRANSLATION OF QUERY AND SEARCH AND RETRIAL OF MULTILINGUALINFORMATION ON A COMPUTER NETWORK”. The patent discloses and teaches amethod for translating a query input by the user in the source language(also called the user's language or the subject language) into thetarget language (also called the object language) and searching andretrieving web documents in the target language and translating the webdocuments into the source language. According to this invention, theuser first inputs a query in a source language through a unit such asthe keyboard. This query is then processed by the server at the backendto extract content word from the input query. The next step takes placeat the dialectal controller, which is present on the server and performsthe function of dialectally standardizing the content word or words soextracted. During this process the user may be prompted for some more soas to refine the search by the user or in case dialectal standardizationcould not be performed using the initial input query. This is followedby the process of pre-search translation, which comprises of translatingthe dialectally standardized word into a target language through atranslator. This process of translation is followed by inputting thetranslated word into a search engine in the target language. Such aninput yields search results in the target language corresponding to thetranslated word. The results so obtained are then displayed in the formof site names (URL) which satisfy the search criteria. All the resultsthus obtained in the target language are then displayed on the userscreen. According to the user's needs such results may then betranslated back either in whole or in part into the source language.Chan's patent aims at assisting a user to search the web by entering aquery in the user's own language, called source language, and returningto the user an entire translation of a targeted web site. In manycircumstances, for a user who has some basic knowledge about the targetlanguage, the translation of an entire document is not necessary.Instead, an instant bilingual annotation on some key words, phrases orsentences would be good enough.

U.S. Pat. No. 6,236,958, issued to Lange et al. discloses a terminologyextraction system which allows for automatic creation of bilingualterminology. The system includes a source text which comprises at leastone sequence of source terms, aligned with a target text which alsocomprises at least one sequence of target terms. A term extractor buildsa network from each source and target sequence wherein each node of thenetwork comprises at least one term and such that each combination ofsource terms is included within one source node and each combination oftarget terms is included within one target node. The term extractorlinks each source node with each target node, and through a flowoptimization method selects relevant links in the resulting network.Once the term extractor has been run on the entire set of alignedsequences, a term statistics circuit computes an association score foreach pair of linked source/target terms, and finally the scored pairs oflinked source/target term that are considered relevant bilingual termsare stored in a bilingual terminology database. The whole process can beiterated in order to improve the strength of the bilingual links.Lange's patent does neither teach a linguistic calibrating mechanismusing statistic abstraction and fuzzy logic, nor a mechanism ofinstantly displaying a bilingual annotation in a callout dynamicallyassociated with the user's mouse pointer.

Accordingly, it would be desirable to provide a system and method forautomatically providing a computer user an artificial intelligence basedbilingual annotation, displayed in a callout associated with the user'smouse pointer, on a piece of textual information contained in a segmentof text adjacent to, or overlaid by, the user's mouse pointer while theuser is reading an electronic document on the computer screen.

It would be further desirable to provide a system and method forautomatically returning a remote online user from a web server anartificial intelligence based bilingual annotation, displayed in acallout associated with the user's mouse pointer, on a piece of textualinformation contained in a segment of text adjacent to, or overlaid by,the user's mouse pointer while the user is viewing the web sitesupported by the web server.

It would be further desirable to provide a subscription based system andmethod for automatically returning a remote online user from athird-party, centralized translation server an artificial intelligencebased bilingual annotation, displayed in a callout associated with theuser's mouse pointer, on a piece of textual information contained in asegment of text adjacent to, or overlaid by, the user's mouse pointerwhile the user is viewing the web site supported by any web server.

SUMMARY OF THE INVENTION

The present invention, defined by the appended claims with the specificembodiments shown in the attached drawings, is directed to a system andmethod that provides a user a bilingual annotation initiated by theuser's mouse pointer. In one preferred embodiment of the invention, itis disclosed a system and method that instantly provides a computer usera bilingual annotation message, contained in a callout associated withthe user's mouse pointer, on a piece of textual information while theuser, who is reading an electronic document displayed on the computerscreen, moves the mouse pointer over, or points the mouse pointer to, asegment of text containing said piece of textual information. Thisembodiment involves a software application which runs on the user'scomputer and operates to perform the following steps:

screen-scraping a segment of text in a first language (object language)which is adjacent to, or overlaid by, the user's mouse pointer;

calibrating the screen-scraped segment of text into a query;

translating the query into a second language (subject language); and

displaying the query and its translation (even other reading aidinformation) in a callout or a virtual bubble closely associated withthe user's mouse pointer.

In another preferred embodiment of the invention, it is disclosed asystem and method that instantly returns to a web user from a backendserver a bilingual annotation message, contained in a callout associatedwith the user's mouse pointer, on a piece of textual information whilethe user, who is reading a web page displayed on a computer screen,moves the mouse pointer over, or points the mouse pointer to, a segmentof text containing said piece of textual information. This embodimentinvolves a software application which runs on the backend server of theweb site and operates to perform the following steps:

screen-scraping a segment of text adjacent to, or overlaid by, theuser's mouse pointer, the segment of text being included in a web pagein an object language;

sending the screen-scraped segment of text to the backend server hostingthe web page;

calibrating the screen-scraped segment of text into a query;

translating the query into a subject language;

returning the user's computer the data required for displaying the queryand its translation (even other reading aid information) in a calloutclosely associated with the user's mouse pointer; and

displaying the callout according to a signal sent from the server.

Yet in another preferred embodiment of the invention, it is disclosed amethod and system that instantly returns a web user from a third-partyserver a bilingual annotation message, contained in a callout associatedwith the user's mouse pointer, on a piece of textual information whilethe user, who is reading a web page or other electronic displayed on acomputer screen, moves the mouse pointer over, or points the mousepointer to, a segment of text containing said piece of textualinformation. This embodiment involves a software application which runson a third-party server and operates to perform the following steps:

screen-scraping a segment of text adjacent to, or overlaid by, theuser's mouse pointer, the segment of text being included in a web pageor other electronic document in an object language;

sending the screen-scraped segment of text to a third-party server whichprovides bilingual annotation service;

calibrating the screen-scraped segment of text into a query;

translating the query into a subject language;

returning the user's computer the data required for displaying the queryand its translation (even other reading aid information) in a calloutclosely associated with the user's mouse pointer; and

displaying the callout according to a signal sent from the server. Theforegoing has outlined rather broadly, the more pertinent and importantfeatures of the present invention. The detailed description of theinvention that follows is offered so that the present contribution tothe art can be more fully appreciated.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more succinct understanding of the nature and goals of the presentinvention, reference should be directed to the following detaileddescription taken in connection with the accompanying drawings in which:

FIG. 1A is a schematic block diagram illustrating a multilinguallinguistic annotation calibration engine (LACE) which runs,independently from any web server, on a computing device according toone preferred embodiment of the invention;

FIG. 2B is a flow diagram further illustrating a process for the LACEaccording to FIG. 1A;

FIG. 2A is a schematic diagram illustrating a system which comprises amultilingual linguistic annotation calibration engine (LACE) which runson a backend server of a web site according to another preferredembodiment of the invention;

FIG. 2B is a block diagram illustrating the operation steps in both theuser's and the backend server's side according to FIG. 2A;

FIG. 2C is a schematic diagram illustrating an exemplary dropdown menufor selecting a subject language to be used in annotation;

FIG. 2D is a schematic diagram illustrating a number of virtual buttons,each of which represents a subject language;

FIG. 2E is a schematic diagram illustrating a rounded rectangularannotation callout;

FIG. 2F is a schematic diagram illustrating a cloud annotation callout;

FIG. 2G is a schematic block diagram further illustrating the preferredembodiment of the invention according to FIG. 2A;

FIG. 3A is a schematic block diagram illustrating a system whichcomprises an instant multilingual linguistic annotation calibrationengine (IM_LACE) which runs on a central translation server whichprovides IM_LACE service on a subscription basis according to anotherpreferred embodiment of the invention; and

FIG. 3B is a flow diagram illustrating a process for providingcentralized instant multilingual LACE service according to the preferredembodiment illustrated in FIG. 3A.

DETAILED DESCRIPTION OF THE INVENTION

With reference to the drawings, the present invention will now bedescribed in detail with regard for the best mode and the preferredembodiments. In its most general form, the invention comprises a programstorage medium readable by a computer, tangibly embodying a program ofinstructions executable by the computer to perform the steps necessaryto provide a user with a bilingual annotation message displayed in acallout associated with the user's mouse pointer.

FIG. 1A is a schematic block diagram illustrating a multilinguallinguistic annotation calibration engine (LACE) 100 according to onepreferred embodiment. Multilingual LACE 100 runs on a computer platform110 which includes one or more central processing units (CPU) 101, arandom memory (RAM) 102, an input/output (I/O) interface 103, anoperating system (OS) 104, and optionally a microinstruction code (MC)105. The multilingual LACE 100 may be part of the microinstruction code(MC) 105 or an application program to be executed via the operatingsystem (OS) 104. Those skilled in the art will readily understand thatmultilingual LACE 100 may be implemented within other systems withoutsubstantial changes.

A user, who is viewing an electronic document in a first language, oftenreferred to as object language, on the computer screen 109, may activatemultilingual LACE at any time. The electronic document can be in anyformat, such as Microsoft Word, Microsoft Excel, Microsoft PowerPoint,PDF, JPEG, etc. When multilingual LACE is activated, the user can set asecond language, often referred to as subject language, to be used forannotation from a language setting 117, which can be a graphical userinterface (GUI) element comprising a dropdown list or a number of icons,each of which represents an option. In the context of this application,the “subject language” means the language, other than the language usedin the target or object document, that the user desires to use forannotating the information contained in the target or object document.Accordingly, the “object language” means the language, other than thesubject language, that is used in the document that the user is readingor viewing. In our example as illustrated in FIG. 1A, the user selectssimplified Chinese as the subject language. From a callout setting 118,the user may configure the parameters structuring and styling a callout,often referred to as a bubble, to be used to display bilingualannotation. The parameters include, but are not limited to, style,shape, font style and size, and background color. The callout setting118, similar to the language setting 117, can be a GUI elementcomprising a dropdown list or a number of icons, each of whichrepresents an option. In one deployment, the language setting 117 andthe callout setting 118 are incorporated into a single GUI 108. Inanother deployment, the language setting 117 and the callout setting 118are coupled to a displayed callout in such a convenient manner that, forexample, these settings are usually hidden but the user may access themby a right-clicking on the callout. Before the user changes thesesettings, they are in the default status or in the status as the userused the application last time.

A callout or a bubble used in this invention is a dynamically createdvisual cue overlaid on the computer screen. Although the style, shape,font style and size as well as background color can be preset by theuser, the content displayed therein is determined by the display module116 based on the outputs of the calibration module 113 and thetranslation module 114. In a bilingual mode, the callout contentprovided by the display module 116 is bilingual. If the user chooses twolanguages at the same time from the language setting 117, the displaycontent will be trilingual. It is possible that the user chooses severallanguages at the same time from the language setting 117 and obtains amultilingual annotation on a query in an object language. Although thecallout or the bubble can be fixed in size, preferably it is adaptiveaccording to the content to be displayed. The term “adaptive” hereinmeans elastic, flexible, scalable, automatically adjusted, to fit thecontent to be displayed. For example, when the query and its translation(and/or even other reading aid information) are very short, the calloutor the bubble is relatively small; otherwise, it can be relativelylarge.

When the user moves her mouse pointer over the electronic documentdisplayed on the computer screen, the mouse pointer initiates ascreen-scraping function 112. The mouse pointer, usually referred to aspointer, is a small bitmap e.g. a small arrow provided by the operatingsystem (OS) 104, that moves on the computer screen in response to themovement of a pointing device, typically a mouse. As the mouse pointermoves, it generates motion events and gives the user feedback. It alsoshows the user which object on the screen will be selected when a mousebutton is clicked, sometimes in combination with a drag action. In thepreferred embodiments of this invention, the mouse pointer is soconfigured that when it moves over or points at a line of text, asegment of text is automatically selected. In other words, the user doesnot need to take click or drag action. Nevertheless, the user can alwaysactivate the manual selection at any time.

Now referring back to FIG. 1A, when the user moves the mouse pointer 111towards a line of text containing “. . . the book titled Living Historywritten by Hillary Rodham Clinton . . . ”, the multilingual LACEapplication screen-scrapes a segment of text from the line. The lengthof the screen-scraped segment of text can be configured according to theuser's needs. Assuming in our example in FIG. 1A, “Living Historywritten by” is screen-scraped and is sent as an input to the calibrationmodule 113. The calibration module 113 standardizes the input into acalibrated query, such as a phrase, a key word, or a sentence, accordingto a number of predefined logic, linguistic and grammatical rules. Thelength of the screen-scraped segment of text can be configured to beadaptive, which means it is elastic, flexible, scalable, automaticallyadjusted. In that case, the user's preferences and the logic, linguisticand grammatical rules used for calibration are applied to segment lengthconfiguration and the screen-scraped text can be directly used as aquery for the translation module 114 because the screen-scraped text isalready calibrated. In either case, the calibration operation isartificial intelligence (AI) based and thus the calibrated query is veryclose to a selection made by a human linguistic expert.

The translation module 114 takes the calibrated query as an input andperforms an AI-based translation by looking up the multilingual database115 following a number of predefined logic, linguistic and grammaticalrules. Because the database 115 and the translation rules reflect thenewest development in the field of machine translation and can beupdated from time to time, the translation made by the translationmodule 114 should be very close to a translation made by a professionaltranslator.

The display module 116 is a multifunctional unit. It accepts the user'scallout setting preferences made from the callout setting 118. It alsocalculates the size of a callout according to the user's preferences andthe character string length for the bilingual annotation containing thecalibrated query in the object language from the calibration module 113and the query's translation from the translation module 114. It “wraps”the query and its translation (and/or even other reading aidinformation) in the callout. It defines the position of the calloutaccording to the mouse pointer's position, the size of the callout andother parameters. Then it sends the data and meta-data to the computerscreen which displays the bilingual annotation callout 119 to the user.

FIG. 1B is a block diagram further illustrating a process for themultilingual LACE according to FIG. 1A. The process includes the stepsof:

Step 121: Activate LACE (LACE can be automatically activated when theuser selects a subject language);

Step 122: Set a subject language to be used for annotating textualinformation in an object language according to the user's selection orthe default selection;

Step 123: Screen-scrape a segment of text which is automaticallyselected when the mouse pointer moves over or points at a line of textincluding the segment of text;

Step 124: Calibrate the screen-scraped text into a query fortranslation;

Step 125: Translate the query into the subject language;

Step 126: Make a callout which fits the query and its translation(and/or even other reading aid information) and wrap them in thecallout; and

Step 127: Display the callout in a position determined by variousparameters such as the mouse pointer's position, the callout's size, thecharacter string length for the bilingual annotation (i.e. the query,its translation, and/or even other reading aid information), andpreferences preset by the user or the default preferences.

Step 128 is performed by the user at any time.

The multilingual LACE described above, with reference to FIG. 1A andFIG. 1B, is preferably deployed as a software program to be distributedto the public. It is also preferably configured to be capable ofscreen-scraping any electronic document displayed on the user's screen.For example, the user can do multilingual LACE on a WORD document, a PDFdocument, or an HTML document on the Internet.

The multilingual LACE can also be incorporated in any document creationsoftware such as WORD or EXCEL. In that case, the user can simplyactivate or deactivate the annotation function from the principalprogram's general menu.

It is also useful to have a simplified version of the multiple LACEprogram embedded in a lightweight device such as a PDA, a cellularphone, or a double-way pager.

In another preferred embodiment, the invention provides a system andmethod for dynamically returning a remote online user a bilingualannotation, displayed in a mouse pointer associated callout, on thetextual information contained in the website. The system, asschematically illustrated in FIG. 2A includes a web server 210 whichsupports a website 211 on the Internet 212. The remote end user 213 logson the Internet 212 by using a browser in her computer and visits awebsite such as the website 211. The website is in an object language,such as English. The multilingual LACE 214 can be activated from the website but runs on the web site server 210. Upon activation of themultilingual LACE 214, the user can obtain bilingual annotation ontextual information in the website by moving her mouse pointer over, orpointing the pointer at, the text that she wants to understand. Forexample, when the user moves the pointer over “Products”, a pop-upcallout 215 comes to the screen. The callout is associated with thepointer such that a visual reference between the callout and the targettext is established. For example, the tail of the annotation callout 215in FIG. 1 points to the text “Products”.

FIG. 2B is a block diagram illustrating the operation steps in both theuser's and the server's side. By entering a URL or by clicking ahyperlink, the user accesses to a website hosted by the web server (Step221). The website is in an object language, such as English. When theuser wants to see bilingual annotation on some words, phrases, orsentences in the website, she needs to activate the multilingual LACE(Step 222) and selects a subject language, such as Chinese, from a list(Step 223). As soon as the subject language is selected, ascreen-scraping means is associated with the user's mouse pointer.Following a number of predefined rules represented by an algorithm, thescreen-scraper, which is a part of the multilingual LACE application,takes a segment of text which falls in a region spatially close to thepointer and sends the scraped segment of text back to the web server viaHTTP (Step 224). Upon standardizing the scraped segment of text into aquery (Step 225), the multilingual LACE in the server side translatesthe query by looking up a powerful multilingual database (Step 226).Then, the web server returns the requested bilingual annotation,including the query and its translation (and/or even other reading aidinformation), together with the meta-data necessary for defining thecallout for the annotation, to the user's computer (Step 227). Theuser's computer displays the returned data on the screen according to asignal sent from the server (Step 228).

The multilingual LACE according to the embodiment illustrated in FIG. 2Aand FIG. 2B is a cross platform application which runs primarily on thebackend server. The application includes an activation means which isimplemented as a graphical user interface embedded in each page of thewebsite. When the user accesses the website, she can activate ordeactivate the multilingual LACE from any page. In one deployment, theuser activates or deactivates the application by clicking an activationbutton. In another deployment, the user activates or deactivates theapplication by choosing from a dropdown menu. Yet in another deployment,the application is automatically deactivated when the user leaves theweb site. These methods for activation and deactivation can be combinedin one way or another as long as it is convenient to the user.

The application also includes a selection means for selecting one ormore subject languages from a list of options. Similar to the activationmeans, the selection means can be deployed as a dropdown menu, a numberof iconic buttons (each of which representative of a language), or anyother elements incorporated in a graphical user interface or a web page.

The activation means and the selection means described above can also beincorporated in one way or another. For example, when the user selects alanguage from a list of options, the multilingual LACE is automaticallyactivated. To deactivate the application, the user may choose“deactivate LACE” from the list or by clicking an icon.

FIG. 2C is a schematic diagram illustrating an exemplary dropdown menufor selecting one or more subject languages to be used in annotation.FIG. 2D is a schematic diagram illustrating a number of virtual buttons,each of which represents a subject language. As an example, assuming theoriginal site language, i.e. the object language is English and Chineseis selected as the subject language, when the user moves the pointerover or points at a phrase or a sentence in the website, there instantlyappears a callout or a “bubble” associated with the pointer. The calloutor the “bubble” contains the phrase or sentence in English and itsChinese translation.

The callout or the “bubble” can be configured in any shape, any color,any background, and any size. In addition, the user can set the fontstyle and size used in the callout or “bubble”, just like setting fontin most of word processing applications and messaging applications. FIG.2E illustrates a rounded rectangular annotation callout, in which font“Time New Roman” is used. FIG. 2F illustrates a cloud annotationcallout, in which font “Courier New” is used.

The difference between a callout and a “bubble” is that the former has abody and a tail, but the latter has a body only. The tail is usefulbecause it is often used as a reference connector between the annotationcallout and the textual information which is annotated. Although acallout is preferably used in various embodiments of this invention, itdoes not deviate from the essence and scope of this invention if someother kind of visual cue such as square, rectangle, circle, bubble, a“kite” or a “halo” is used to display the returned annotation message.

As an example, the callout can be configured to a fixed size. In thiscase, only a limited number of characters can be displayed in thecallout. When the pointer moves, the callout, like a moving window, onlyshows the bilingual annotation on the words which are spatially closerto the pointer. The annotation on the words which are getting fartherfrom the pointer automatically disappears from the callout.

As another example, the user can configure a sentence-by-sentencetranslation scheme. In this case, when the pointer moves over asentence, the translation of the sentence is displayed in the bubble.Because some sentences are long and some are very short, a flexiblebubble is most appropriate.

The multilingual LACE application scrapes text from the screen followinga number of predefined rules, for examples: only the text in a line mostclose to the pointer is scraped; one inch of the segment in the left (orright) of the pointer is scraped; only the segment one inch to the rightand one inch to the left of the pointer is scraped; or a whole isscraped, etc.

Now turning to FIG. 2G, which is schematic block diagram furtherillustrating the preferred embodiment of the invention according to FIG.1A. When the user points the mouse pointer 241 to the screen text “Portof Oakland”, the screen-scraper 242, which is part of the multilingualLACE application, makes a screen-scraping operation. The screen-scrapedsegment of text is sent to the server 240 via HTTP, which includes acalibration module 243, a translation module 244 coupled to amultilingual database 245, and a callout making module 246. Thecalibration module 243 performs a number of logic, linguistic andgrammatical operations to calibrate the screen-scraped segment of textinto a standardized query. The translation module 244 translates thequery, by looking up the powerful multilingual database 245 andperforming relevant linguistic and grammatical calculations, into arepresentation in a subject language selected by the user from thelanguage selection interface 247 which is available in the website 250.Based on the user's preferences and relevant calculations, the calloutmaking module 246 determines the size, style, shape, font style and sizeof the callout required to display the annotation which includes thequery in the object language and the query's translation in one or moresubject languages. Preferably, a bilingual representation is needed. Thestyle, font and background color, etc. for the callout 249 can beconfigured by the user using the callout setting interface 248 which isavailable in the website 250.

The calibration module 243 may perform functions such as dialectal wordlookup, collection of spontaneous innovation, lexical diffusion,statistical abstraction and fuzzy logic, parsing, complex sentencesdecomposition, etc. The logic, linguistic and grammatical rules used bythe calibration module 243 include, but are not limited to thefollowing: Identify a complete sentence by extracting the text betweenany two neighboring periods (“.”), or between one period (“.”) and anexclamation mark (“!”), or between one period (“.”) and a question mark(“?”), in the screen-scraped text; If no complete sentence isidentified, identify a key phrase by ignoring pronouns, copulas, etc.

The callout making module 246 not only determines the size of thecallout 249, but also determines the callout's position relative to themouse pointer 241. As illustrated in FIG. 2H, when the mouse pointer isvery close to the right edge of the page, the callout is placed in thepointer's left side so as to keep the callout within the page.Similarly, when the mouse pointer is very close to the left edge of thepage, the mouse pointer is placed in the right of the mouse pointer;when the mouse pointer is very close to the upper edge of the page, thecallout is placed no higher than the mouse pointer; and when the pointeris very close to the bottom of the page, the callout is placed no lowerthan the mouse pointer.

Note that the translation module 244 performs translation based on a setof predefined logic, linguistic and grammatical rules which are specificto the language selected. The more sophisticated the rules are, the moreprecise the translation is. In addition, the translation module 244 isartificial intelligence (AI) based. For example, it is empowered withvalence features, collocational probabilities, statistic abstraction aswell as fuzzy logic.

The multilingual LACE described above, with reference to FIG. 2A-FIG.2H, is preferably deployed as a software application specific to the website hosted by the web site server. It is also preferably configured tobe capable of screen-scraping information on the web site only. In otherwords, the user can not activate the multilingual LACE from one site anduse it on other documents other than these posted in the web site.Otherwise, the system would become a free carrier.

Yet in another preferred embodiment of the invention as illustrated inFIG. 3A, an instant multilingual LACE service, called IM_LACE, isprovided from a central translation server 310 using an instantmessaging (IM) framework, which is either an independent IM system orincorporated in an existing IM system such as NetMeeting, MSN Messenger,Yahoo! Messenger, AIM, etc. The data exchange between users and thecentral translation server 310 is supported by web service interfaces,such as SOAP/XML/HTTP, and the related protocols.

Preferably, IM_LACE service is subscription based. An individual user,such as user 312 or user 317 subscribes the service by registration anddownloading the IM_LACE client application. When the client applicationis downloaded, the user can log in the service and use it online againstany electronic document. The client application can be configured toexecute the calibration and callout making tasks but leaves thetranslation, which usually requires a large database, for the centralserver 310. In FIG. 3A, user 316 is using the IM_LACE service in the IMsession 317. Similarly, user 312 in IM session 315 is using the IM_LACEservice to view a web site supported by the qN site server 311 on theInternet 318.

FIG. 3B is a block diagram illustrating a process according to theembodiment of FIG. 3A. The process includes the steps of:

Step 321: Log on (activate) the IM_LACE system;

Step 322: Screen-scrape a segment of text adjacent to, or overlaid by,the user's mouse pointer, the segment of text being included in a webpage or other electronic document in an object language;

Step 323: Calibrate the screen-scraped segment of text into a query;

Step 324: Send the query to the centralized translation server;

Step 325: Return translation to the IM_LACE client application in theuser's local computer; and

Step 326: Display the query and its translation (and/or even otherreading aid information) in a callout closely associated with the user'smouse pointer.

The advantages of the invention described above are numerous. First, bycalibrating the screen-scraped text using an AI-based module such as thecalibration module 243 in FIG. 2G, a more content-relevant annotation ismade available.

Second, the translation module is also AI-based. By adopting highlysophisticated AI translation technology, the translation is as much asclose to human expert translation.

Third, the annotation is dynamic because the displaying callout orbubble is associated with the user's mouse pointer and the displayedbilingual annotation is specifically on the segment of textualinformation spatially close to the mouse pointer.

Fourth, the system is user-friendly because a user can easily set thestyle, font and background color etc. of the callout or bubble.

Fifth, as an elegant device by providing instant, pop-up, contextualizedtranslation of key information to foreigners without going into expensecreating a site in a whole different language, LACE helps maintainintegrity and centrality of the principal site. Foreigners only have toselect which subject language they want to activate.

Although the invention is described herein with reference to thepreferred embodiment, one skilled in the art will readily appreciatethat other applications may be substituted for those set forth hereinwithout departing from the spirit and scope of the present invention.

Accordingly, the invention should only be limited by the Claims includedbelow.

1. A system for providing a user with bilingual annotation on a piece oftextual information in a first language contained in an electronicdocument displayed in the user's screen, the system comprising aprocessor which is configured to: screen-scrape a segment of textadjacent to, or overlaid by, the user's pointer; calibrate saidscreen-scraped segment of text into a query according to one or morelogic, linguistic and/or grammatical rules; translate said query into asecond language by looking up a database and applying a set of logic,linguistic and grammatical rules; and display a visual cue on the user'sscreen, said visual cue containing said query, said query's translationand/or other reading aid information.
 2. The system of claim 1, whereinsaid segment of text is fixed in length.
 3. The system of claim 1,wherein the length of said segment of text is automatically adjustedaccording to one or more logic, linguistic and/or grammatical rules. 4.The system of claim 1, wherein said visual cue is dynamically associatedwith the user's pointer.
 5. The system of claim 4, wherein said visualcue comprises a tail which approximately overlaps with the user'spointer.
 6. The system of claim 1, wherein said visual cue is fixed insize.
 7. The system of claim 1, wherein said visual cue is adaptive tofit the content therein.
 8. A computer usable medium containinginstructions in computer readable form for carrying out a process forproviding a user with bilingual annotation on a piece of textualinformation in a first language contained in an electronic documentdisplayed in the user's screen, said process comprising the steps of:screen-scraping a segment of text adjacent to, or overlaid by, theuser's pointer, calibrating said screen-scraped segment of text into aquery; translating said query into a second language; and displaying acallout on the user's screen, said callout containing said query, saidquery's translation and/or other reading aid information.
 9. Thecomputer usable medium of claim 8, wherein said segment of text is fixedin length.
 10. The computer usable medium of claim 8, wherein the lengthsaid segment of text is automatically adjusted according to one or morelogic, linguistic and/or grammatical rules.
 11. The computer usablemedium of claim 8, wherein said callout is dynamically associated withthe user's pointer.
 12. The computer usable medium of claim 11, whereinsaid callout's tail approximately overlaps with the user's pointer. 13.The computer usable medium of claim 8, wherein said callout is fixed insize.
 14. The computer usable medium of claim 8, wherein said callout isadaptive to fit the content therein.
 15. A method for providing a userwith bilingual annotation on a piece of textual information in a firstlanguage contained in an electronic document displayed in the user'sscreen, comprising the steps of: screen-scraping a segment of textadjacent to, or overlaid by, the user's pointer; calibrating saidscreen-scraped segment of text into a query according to one or morerules; translating said query into a second language by looking up adatabase and applying a set of logic, linguistic and grammatical rules;and displaying an annotation callout on the user's screen, saidannotation callout containing said query, said query's translationand/or other reading aid information.
 16. The method of claim 15,wherein said segment of text is fixed in length.
 17. The method of claim15, wherein the length said segment of text is automatically adjustedaccording to one or more logic, linguistic and/or grammatical rules. 18.The method of claim 15, wherein said callout is dynamically associatedwith the user's pointer.
 19. The method of claim 18, wherein saidcallout's tail approximately overlaps with the user's pointer.
 20. Themethod of claim 15, wherein said callout is fixed in size.
 21. Themethod of claim 15, wherein said callout is adaptive to fit the contenttherein.
 22. A system for returning to a remote user from a web server abilingual annotation on a piece of textual information in a firstlanguage contained in a website supported by the web server, said systemcomprising an application which operates to: screen-scrape a segment oftext adjacent to, or overlaid by, the user's pointer; calibrate saidscreen-scraped segment of text into a query; translate said query into asecond language; and send a signal to display said query, said query'stranslation and/or other reading aid information in a visual cue on theuser's screen.
 23. The system of claim 22, wherein said applicationcomprises a graphical user interface embedded in each page of said website, said graphical user interface comprising: means for activation ordeactivation of said application; and means for selecting said secondlanguage from a list of languages.
 24. The system of claim 23, whereinsaid application is automatically activated when said second language isselected.
 25. The system of claim 22, wherein said segment of text isfixed in length.
 26. The system of claim 22, wherein the length of saidsegment of text is automatically adjusted according to one or morelogic, linguistic and/or grammatical rules.
 27. The system of claim 22,wherein said visual cue's position is dynamically associated with theuser's pointer.
 28. The system of claim 27, wherein said visual cuecomprises a tail which approximately overlaps with the user's pointer.29. The system of claim 28, wherein said visual cue is fixed in size.30. The system of claim 22, wherein said visual cue is adaptive to fitthe content therein.
 31. The system of claim 23, wherein said graphicaluser interface further comprises: means for setting parameters of saidvisual cue.
 32. A method for returning to a remote user from a webserver a bilingual annotation on a piece of textual information in afirst language contained in a website supported by the web server,comprising the steps of: screen-scraping a segment of text adjacent to,or overlaid by, the user's pointer; sending said screen-scraped segmentof text to the web server; calibrating said screen-scraped segment oftext into a query according to one or more rules; translating said queryinto a second language by looking up a database and applying a set oflogic, linguistic and grammatical rules; and returning said query alongwith said query's translation to the user's computer; and sending asignal to display a callout containing said query, said query'stranslation and/or other reading aid information on the user's screen.33. The method of claim 32, wherein said application comprises agraphical user interface embedded in each page of said web site, saidgraphical user interface comprising: means for activation ordeactivation of said application; and means for selecting said secondlanguage from a list of languages.
 34. The method of claim 33, whereinsaid application is automatically activated when said second language isselected.
 35. The method of claim 32, wherein said segment of text isfixed in length.
 36. The method of claim 32, wherein the length of saidsegment of text is automatically adjusted according to one or morelogic, linguistic and/or grammatical rules.
 37. The method of claim 32,wherein said callout's position is dynamically associated with theuser's pointer.
 38. The method of claim 37, wherein said callout's tailapproximately overlaps with the user's pointer.
 39. The method of claim32, wherein said callout is fixed in size.
 40. The method of claim 32,wherein said callout is adaptive to fit the content therein.
 41. Themethod of claim 32, wherein said graphical user interface furthercomprises: means for setting parameters of said callout.
 42. A systemfor providing real-time multilingual annotation service over a globalnetwork from a server to a user, said system comprising: (a) a clientapplication which runs on the user' computer, said client applicationbeing operable to: screen-scrape a segment of text in a first language,said segment of text being adjacent to, or overlaid by, the user'spointer; calibrate said screen-scraped segment of text into a query;send said query to the server; and display an annotation callout whichcontains said query and the translation of said query returned from theserver; and (b) a server application which runs on the server, saidserver application being operable to: translate said query into a secondlanguage by looking up a database and applying a set of logic,linguistic and grammatical rules; and return the translation of saidquery to the client application.
 43. The system of claim 42, whereinsaid segment of text is fixed in length.
 44. The system of claim 42,wherein said segment of text is automatically adjusted according to oneor more logic, linguistic and grammatical rules.
 45. The system of claim42, wherein said callout is dynamically associated with the user'spointer.
 46. The system of claim 45, wherein said callout's tailapproximately overlaps with the user's pointer.
 47. The system of claim42, wherein said callout is fixed in size.
 48. The system of claim 42,wherein said callout is adaptive to fit the content therein.
 49. Amethod for providing real-time multilingual annotation service over aglobal network from a server to a user, said method comprising:screen-scraping a segment of text in a first language, said segment oftext being adjacent to, or overlaid by, the user's pointer; calibratingsaid screen-scraped segment of text into a query; sending said query tothe server; translating said query at the server into a second languageby looking up a database and applying a set of logic, linguistic andgrammatical rules; returning the translation of said query to the user'scomputer; and displaying an annotation callout which contains saidquery, the translation of said query, and/or other reading aidinformation, returned from the server.
 50. The method of claim 49,wherein said segment of text is fixed in length.
 51. The method of claim49, wherein the length of said segment of text is automatically adjustedaccording to one or more logic, linguistic and/or grammatical rules. 52.The method of claim 49, wherein said callout is dynamically associatedwith the user's pointer.
 53. The method of claim 52, wherein saidcallout's tail approximately overlaps with the user's pointer.
 54. Themethod of claim 49, wherein said callout is fixed in size.
 55. Themethod of claim 49, wherein said callout is adaptive to fit the contenttherein.
 56. A system for providing an annotation on a piece of textualinformation in a first language contained in an electronic documentstored in a server communicatively connected to a client via a network,the system comprising a processor configured to: receive from the clientdata identifying said piece of textual information; calibrate saididentified textual information into a query according to one or morelogic, linguistic and/or grammatical rules; translate said query into asecond language by looking up a database and applying a set of logic,linguistic and grammatical rules; and forward to the client atranslation of said query.
 57. A computer usable medium containinginstructions in computer readable form for carrying out a process forproviding a user with bilingual annotation on a piece of textualinformation in a first language contained in an electronic documentdisplayed in the user's screen, said process comprising: receiving dataidentifying said piece of textual information; calibrating said piece oftextual information into a query; translating said query into a secondlanguage; and forwarding said translated query to the user.
 58. A methodfor providing a user with bilingual annotation on a piece of textualinformation in a first language contained in an electronic documentdisplayed in the user's screen, said method comprising: receiving dataidentifying said piece of textual information; calibrating said piece oftextual information into a query; translating said query into a secondlanguage; and forwarding said translated query to the user.